{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "4239cd37",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Rank 1: gpt-5-nano\n"
     ]
    }
   ],
   "source": [
    "results_dict = json.loads(results)\n",
    "ranks = results_dict[\"results\"]\n",
    "for index, result in enumerate(ranks):\n",
    "    competitor = competitors[int(result)-1]\n",
    "    print(f\"Rank {index+1}: {competitor}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "b025ec1e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"results\": [\"1\"]}\n"
     ]
    }
   ],
   "source": [
    "openai = OpenAI()\n",
    "response = openai.chat.completions.create(\n",
    "    model=\"gpt-5-mini\",\n",
    "    messages=judge_messages,\n",
    ")\n",
    "results = response.choices[0].message.content\n",
    "print(results)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "d2f44294",
   "metadata": {},
   "outputs": [],
   "source": [
    "judge_messages = [{\"role\": \"user\", \"content\": judge}]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "b21ad62d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "You are judging a competition between 1 competitors.\n",
      "Each model has been given this question:\n",
      "\n",
      "How many bricks would i need to build a house of 1000 square feet?\n",
      "\n",
      "Your job is to evaluate each response for clarity and strength of argument, and rank them in order of best to worst.\n",
      "Respond with JSON, and only JSON, with the following format:\n",
      "{\"results\": [\"best competitor number\", \"second best competitor number\", \"third best competitor number\", ...]}\n",
      "\n",
      "Here are the responses from each competitor:\n",
      "\n",
      "# Response from competitor 1\n",
      "\n",
      "Short answer: it depends on how thick the walls are, how tall they are, and how much opening (doors/windows) you have. A rough ballpark for a typical one‑brick‑thick exterior wall on a 1000 sq ft house is about 6,000–7,000 bricks (before accounting for openings, waste, etc.). If you use two wythes (a double brick wall), it would be about double that.\n",
      "\n",
      "Here’s a simple way to estimate it and a sample calculation:\n",
      "\n",
      "- Step 1: Decide wall area\n",
      "  - Example footprint: about 31.6 ft by 31.6 ft (roughly 1000 sq ft).\n",
      "  - Perimeter ≈ 126.4 ft.\n",
      "  - Exterior wall height: 8 ft (common) or 9 ft (taller rooms).\n",
      "  - Exterior wall area ≈ perimeter × height = 126.4 × 8 ≈ 1,011 ft².\n",
      "\n",
      "- Step 2: Bricks per square foot of wall\n",
      "  - For a standard 7 5/8\" x 2 1/4\" brick with ~3/8\" mortar joints, you get about 6.8 bricks per square foot of wall surface.\n",
      "\n",
      "- Step 3: Raw brick count\n",
      "  - 1,011 ft² × 6.8 ≈ 6,900 bricks.\n",
      "\n",
      "- Step 4: account for openings (windows/doors)\n",
      "  - If openings remove about 15–20% of wall area, bricks drop to roughly 5,700–6,000.\n",
      "\n",
      "- Step 5: add waste/overage\n",
      "  - Add about 5–10% extra for cut bricks, waste, breakage: say 6,000–6,600 bricks total.\n",
      "\n",
      "Notes and variations:\n",
      "- If you build a two-story house with the same footprint, wall area roughly doubles (more brick needed).\n",
      "- If you use a double brick wall (two wythes), multiply by about 2.\n",
      "- If you’re planning brick veneer on a wood frame, you still count bricks for the veneer, but the structure isn’t the same as a full load-bearing brick wall.\n",
      "- Interior walls are often not brick; this estimate is for exterior walls or a full brick exterior if that’s what you’re building.\n",
      "\n",
      "If you can share:\n",
      "- floor plan shape or dimensions (length and width or a rough footprint),\n",
      "- wall height (8 ft, 9 ft, etc.),\n",
      "- are the walls single brick thick or double,\n",
      "- approximate window/door area percentage,\n",
      "\n",
      "I can give you a more precise number.\n",
      "\n",
      "\n",
      "\n",
      "Now respond with the JSON with the ranked order of the competitors, nothing else. Do not include markdown formatting or code blocks.\n"
     ]
    }
   ],
   "source": [
    "print(judge)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "e8aadb12",
   "metadata": {},
   "outputs": [],
   "source": [
    "judge = f\"\"\"You are judging a competition between {len(competitors)} competitors.\n",
    "Each model has been given this question:\n",
    "\n",
    "{question}\n",
    "\n",
    "Your job is to evaluate each response for clarity and strength of argument, and rank them in order of best to worst.\n",
    "Respond with JSON, and only JSON, with the following format:\n",
    "{{\"results\": [\"best competitor number\", \"second best competitor number\", \"third best competitor number\", ...]}}\n",
    "\n",
    "Here are the responses from each competitor:\n",
    "\n",
    "{together}\n",
    "\n",
    "Now respond with the JSON with the ranked order of the competitors, nothing else. Do not include markdown formatting or code blocks.\"\"\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "b8620c42",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# Response from competitor 1\n",
      "\n",
      "Short answer: it depends on how thick the walls are, how tall they are, and how much opening (doors/windows) you have. A rough ballpark for a typical one‑brick‑thick exterior wall on a 1000 sq ft house is about 6,000–7,000 bricks (before accounting for openings, waste, etc.). If you use two wythes (a double brick wall), it would be about double that.\n",
      "\n",
      "Here’s a simple way to estimate it and a sample calculation:\n",
      "\n",
      "- Step 1: Decide wall area\n",
      "  - Example footprint: about 31.6 ft by 31.6 ft (roughly 1000 sq ft).\n",
      "  - Perimeter ≈ 126.4 ft.\n",
      "  - Exterior wall height: 8 ft (common) or 9 ft (taller rooms).\n",
      "  - Exterior wall area ≈ perimeter × height = 126.4 × 8 ≈ 1,011 ft².\n",
      "\n",
      "- Step 2: Bricks per square foot of wall\n",
      "  - For a standard 7 5/8\" x 2 1/4\" brick with ~3/8\" mortar joints, you get about 6.8 bricks per square foot of wall surface.\n",
      "\n",
      "- Step 3: Raw brick count\n",
      "  - 1,011 ft² × 6.8 ≈ 6,900 bricks.\n",
      "\n",
      "- Step 4: account for openings (windows/doors)\n",
      "  - If openings remove about 15–20% of wall area, bricks drop to roughly 5,700–6,000.\n",
      "\n",
      "- Step 5: add waste/overage\n",
      "  - Add about 5–10% extra for cut bricks, waste, breakage: say 6,000–6,600 bricks total.\n",
      "\n",
      "Notes and variations:\n",
      "- If you build a two-story house with the same footprint, wall area roughly doubles (more brick needed).\n",
      "- If you use a double brick wall (two wythes), multiply by about 2.\n",
      "- If you’re planning brick veneer on a wood frame, you still count bricks for the veneer, but the structure isn’t the same as a full load-bearing brick wall.\n",
      "- Interior walls are often not brick; this estimate is for exterior walls or a full brick exterior if that’s what you’re building.\n",
      "\n",
      "If you can share:\n",
      "- floor plan shape or dimensions (length and width or a rough footprint),\n",
      "- wall height (8 ft, 9 ft, etc.),\n",
      "- are the walls single brick thick or double,\n",
      "- approximate window/door area percentage,\n",
      "\n",
      "I can give you a more precise number.\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(together)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "424c29a8",
   "metadata": {},
   "outputs": [],
   "source": [
    "together = \"\"\n",
    "for index, answer in enumerate(answers):\n",
    "    together += f\"# Response from competitor {index+1}\\n\\n\"\n",
    "    together += answer + \"\\n\\n\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "ca981314",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Competitor: gpt-5-nano\n",
      "\n",
      "Short answer: it depends on how thick the walls are, how tall they are, and how much opening (doors/windows) you have. A rough ballpark for a typical one‑brick‑thick exterior wall on a 1000 sq ft house is about 6,000–7,000 bricks (before accounting for openings, waste, etc.). If you use two wythes (a double brick wall), it would be about double that.\n",
      "\n",
      "Here’s a simple way to estimate it and a sample calculation:\n",
      "\n",
      "- Step 1: Decide wall area\n",
      "  - Example footprint: about 31.6 ft by 31.6 ft (roughly 1000 sq ft).\n",
      "  - Perimeter ≈ 126.4 ft.\n",
      "  - Exterior wall height: 8 ft (common) or 9 ft (taller rooms).\n",
      "  - Exterior wall area ≈ perimeter × height = 126.4 × 8 ≈ 1,011 ft².\n",
      "\n",
      "- Step 2: Bricks per square foot of wall\n",
      "  - For a standard 7 5/8\" x 2 1/4\" brick with ~3/8\" mortar joints, you get about 6.8 bricks per square foot of wall surface.\n",
      "\n",
      "- Step 3: Raw brick count\n",
      "  - 1,011 ft² × 6.8 ≈ 6,900 bricks.\n",
      "\n",
      "- Step 4: account for openings (windows/doors)\n",
      "  - If openings remove about 15–20% of wall area, bricks drop to roughly 5,700–6,000.\n",
      "\n",
      "- Step 5: add waste/overage\n",
      "  - Add about 5–10% extra for cut bricks, waste, breakage: say 6,000–6,600 bricks total.\n",
      "\n",
      "Notes and variations:\n",
      "- If you build a two-story house with the same footprint, wall area roughly doubles (more brick needed).\n",
      "- If you use a double brick wall (two wythes), multiply by about 2.\n",
      "- If you’re planning brick veneer on a wood frame, you still count bricks for the veneer, but the structure isn’t the same as a full load-bearing brick wall.\n",
      "- Interior walls are often not brick; this estimate is for exterior walls or a full brick exterior if that’s what you’re building.\n",
      "\n",
      "If you can share:\n",
      "- floor plan shape or dimensions (length and width or a rough footprint),\n",
      "- wall height (8 ft, 9 ft, etc.),\n",
      "- are the walls single brick thick or double,\n",
      "- approximate window/door area percentage,\n",
      "\n",
      "I can give you a more precise number.\n"
     ]
    }
   ],
   "source": [
    "# It's nice to know how to use \"zip\"\n",
    "for competitor, answer in zip(competitors, answers):\n",
    "    print(f\"Competitor: {competitor}\\n\\n{answer}\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "8e6b775a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['gpt-5-nano']\n",
      "['Short answer: it depends on how thick the walls are, how tall they are, and how much opening (doors/windows) you have. A rough ballpark for a typical one‑brick‑thick exterior wall on a 1000 sq ft house is about 6,000–7,000 bricks (before accounting for openings, waste, etc.). If you use two wythes (a double brick wall), it would be about double that.\\n\\nHere’s a simple way to estimate it and a sample calculation:\\n\\n- Step 1: Decide wall area\\n  - Example footprint: about 31.6 ft by 31.6 ft (roughly 1000 sq ft).\\n  - Perimeter ≈ 126.4 ft.\\n  - Exterior wall height: 8 ft (common) or 9 ft (taller rooms).\\n  - Exterior wall area ≈ perimeter × height = 126.4 × 8 ≈ 1,011 ft².\\n\\n- Step 2: Bricks per square foot of wall\\n  - For a standard 7 5/8\" x 2 1/4\" brick with ~3/8\" mortar joints, you get about 6.8 bricks per square foot of wall surface.\\n\\n- Step 3: Raw brick count\\n  - 1,011 ft² × 6.8 ≈ 6,900 bricks.\\n\\n- Step 4: account for openings (windows/doors)\\n  - If openings remove about 15–20% of wall area, bricks drop to roughly 5,700–6,000.\\n\\n- Step 5: add waste/overage\\n  - Add about 5–10% extra for cut bricks, waste, breakage: say 6,000–6,600 bricks total.\\n\\nNotes and variations:\\n- If you build a two-story house with the same footprint, wall area roughly doubles (more brick needed).\\n- If you use a double brick wall (two wythes), multiply by about 2.\\n- If you’re planning brick veneer on a wood frame, you still count bricks for the veneer, but the structure isn’t the same as a full load-bearing brick wall.\\n- Interior walls are often not brick; this estimate is for exterior walls or a full brick exterior if that’s what you’re building.\\n\\nIf you can share:\\n- floor plan shape or dimensions (length and width or a rough footprint),\\n- wall height (8 ft, 9 ft, etc.),\\n- are the walls single brick thick or double,\\n- approximate window/door area percentage,\\n\\nI can give you a more precise number.']\n"
     ]
    }
   ],
   "source": [
    "print(competitors)\n",
    "print(answers)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "73cc8050",
   "metadata": {},
   "outputs": [],
   "source": [
    "# YOU CAN ADD MORE MODELS TO TEST HERE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "271008aa",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Short answer: it depends on how thick the walls are, how tall they are, and how much opening (doors/windows) you have. A rough ballpark for a typical one‑brick‑thick exterior wall on a 1000 sq ft house is about 6,000–7,000 bricks (before accounting for openings, waste, etc.). If you use two wythes (a double brick wall), it would be about double that.\n",
       "\n",
       "Here’s a simple way to estimate it and a sample calculation:\n",
       "\n",
       "- Step 1: Decide wall area\n",
       "  - Example footprint: about 31.6 ft by 31.6 ft (roughly 1000 sq ft).\n",
       "  - Perimeter ≈ 126.4 ft.\n",
       "  - Exterior wall height: 8 ft (common) or 9 ft (taller rooms).\n",
       "  - Exterior wall area ≈ perimeter × height = 126.4 × 8 ≈ 1,011 ft².\n",
       "\n",
       "- Step 2: Bricks per square foot of wall\n",
       "  - For a standard 7 5/8\" x 2 1/4\" brick with ~3/8\" mortar joints, you get about 6.8 bricks per square foot of wall surface.\n",
       "\n",
       "- Step 3: Raw brick count\n",
       "  - 1,011 ft² × 6.8 ≈ 6,900 bricks.\n",
       "\n",
       "- Step 4: account for openings (windows/doors)\n",
       "  - If openings remove about 15–20% of wall area, bricks drop to roughly 5,700–6,000.\n",
       "\n",
       "- Step 5: add waste/overage\n",
       "  - Add about 5–10% extra for cut bricks, waste, breakage: say 6,000–6,600 bricks total.\n",
       "\n",
       "Notes and variations:\n",
       "- If you build a two-story house with the same footprint, wall area roughly doubles (more brick needed).\n",
       "- If you use a double brick wall (two wythes), multiply by about 2.\n",
       "- If you’re planning brick veneer on a wood frame, you still count bricks for the veneer, but the structure isn’t the same as a full load-bearing brick wall.\n",
       "- Interior walls are often not brick; this estimate is for exterior walls or a full brick exterior if that’s what you’re building.\n",
       "\n",
       "If you can share:\n",
       "- floor plan shape or dimensions (length and width or a rough footprint),\n",
       "- wall height (8 ft, 9 ft, etc.),\n",
       "- are the walls single brick thick or double,\n",
       "- approximate window/door area percentage,\n",
       "\n",
       "I can give you a more precise number."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model_name = \"gpt-5-nano\"\n",
    "\n",
    "response = openai.chat.completions.create(model=model_name, messages=messages)\n",
    "answer = response.choices[0].message.content\n",
    "\n",
    "display(Markdown(answer))\n",
    "competitors.append(model_name)\n",
    "answers.append(answer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "2777156d",
   "metadata": {},
   "outputs": [],
   "source": [
    "competitors = []\n",
    "answers = []\n",
    "messages = [{\"role\": \"user\", \"content\": question}]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "b07500f0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "How many bricks would i need to build a house of 1000 square feet?\n"
     ]
    }
   ],
   "source": [
    "openai = OpenAI()\n",
    "response = openai.chat.completions.create(\n",
    "    model=\"gpt-5-mini\",\n",
    "    messages=messages,\n",
    ")\n",
    "question = response.choices[0].message.content\n",
    "print(question)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "d77f309a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'role': 'user',\n",
       "  'content': 'How many bricks would i need to build a house of 1000 square feet. Answer only with the question, no explanation.'}]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "messages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "8bcd687c",
   "metadata": {},
   "outputs": [],
   "source": [
    "request = \"How many bricks would i need to build a house of 1000 square feet. \"\n",
    "request += \"Answer only with the question, no explanation.\"\n",
    "messages = [{\"role\": \"user\", \"content\": request}]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "86e9eda3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "OpenAI API Key exists and begins sk-proj-\n"
     ]
    }
   ],
   "source": [
    "\n",
    "openai_api_key = os.getenv('OPENAI_API_KEY')\n",
    "\n",
    "\n",
    "if openai_api_key:\n",
    "    print(f\"OpenAI API Key exists and begins {openai_api_key[:8]}\")\n",
    "else:\n",
    "    print(\"OpenAI API Key not set\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "232aa8b4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "load_dotenv(override=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "576ab4d9",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import json\n",
    "from dotenv import load_dotenv\n",
    "from openai import OpenAI\n",
    "from anthropic import Anthropic\n",
    "from IPython.display import Markdown, display"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
